Known types of standard disk controllers do not support data compression although more recent disk controllers, organized in a log-structured manner, can support data compression. However, the operation of conventional log-structured disk controllers may cause sequentially stored data to become stored discontiguously on the disk. This results in a loss of locality of reference and leads to a degradation in performance for some workloads, such as database applications.
A log-structured file system (hereinafter referred to as LSFS) is described by M. Rosenblum and John K. Ousterhout in an article entitled "The Design and Implementation of a Log-Structured File System", ACM Transactions on Computer Systems, Vol. 10, No. 1, February 1992, pages 26-52.
Briefly, the LSFS is a technique for disk storage management wherein all modifications to a disk are written sequentially to a log-like file structure. The log-like file structure is the only structure on the disk, and it contains indexing information so that the files can be read back from the log in an efficient manner.
An aspect of the LSFS approach is that large free areas are maintained on the disk in order to speed-up the write process. To maintain the large free areas, the log is divided into segments, and a segment cleaner (garbage collector) is employed to compress live information from heavily fragmented segments, thereby freeing up segments for subsequent writes.
A goal of the LSFS is to improve the efficiency of disk writes by utilizing a larger percentage of the disk bandwidth than other disk management techniques. That is, instead of making a large number of small writes to the disk, the data is instead collected in the storage subsystem cache or buffers, and the file cache is then written out to the disk in a single large I/O (disk write) operation. The physical writing of the segment, however, can proceed in increments.
As was noted above, one problem that arises from the use of such a LSFS is that compressed/compacted data can be scattered over multiple disk locations, thus reducing seek affinity and increasing response time. This problem can be especially troublesome when records are read from the disk, are updated and compressed, and then written back to the disk.
More specifically, read and write requests to disk controllers are typically accompanied by the physical address(es) to or from which data is to be read or written. For example, read/write requests to well known SCSI-type disk controllers are accompanied by the starting physical sector number and the number of sectors to be read/written. In traditional disk controllers, a write operation operates to send uncompressed data to the controller, and the uncompressed data is stored within the indicated sectors, thereby replacing the previous contents of the indicated sectors.
A log-structured disk controller that supports data compression, on the other hand, operates as follows. In that data compression, more particularly the data compression ratio, is data dependent, writes to the disk controller are not written in place, but are instead written to new locations on disk that were previously empty. The disk controller divides the disk into segments, some of which are kept "empty". New writes from the system are written into sectors located within empty segments. As a result, each write or update of data causes the data to be written to new physical locations, and the previous physical locations of the data are subsequently garbage collected and reused for future writes. Furthermore, in a log-structured disk controller a directory must be maintained to map the addresses used by the system to the physical addresses at which the data is actually stored within the log structure.
This is clearly different than the operation of a conventional disk controller, where the data is always updated in place, and where the address specified in the request from the system is the same as the address at which the data is stored on the disk(s).
To summarize, known-types of conventional log-structured disk controllers do not update data in place, and they maintain a directory to find the current location of any piece of data.
In that data compression techniques are sensitive to the data being compressed, even a relatively small change to a record can result in the compressed image growing in size relative to the previous, unmodified compressed image. Thus, even if an update in place mechanism were available with the conventional log-structured disk controllers, if the compressed data image grows in size there is a high probability that it will not fit back into its previous position within the LSFS. In this case it would still be necessary to re-write the compressed data image to another physical location within the disk. As a result, a once physically and logically contiguous sequence of records would still be fragmented and scattered, thereby increasing seek times for subsequent accesses to the records.
It is also known in the art to employ, instead of one large disk (also referred to as a Single Large Expensive Disk or SLED), a Redundant Array of Inexpensive Disks (RAID), as described by D. A. Patterson, G. Gibson, and R. H. Katz in an article entitled "A Case for Redundant Arrays of Inexpensive Disks (RAID)" ACM SIGMOD Conference, Chicago, Ill., Jun. 1-3, 1988, pages 109-116. An advantage of the RAID approach is that it enables the disk subsystem of a data processor to keep pace with the continuing improvements in processor speed and main memory density. However, the authors show that the Mean Time To Failure (MTTF) of the RAID storage system is given by the MTTF of a single disk divided by the total number of disks in the array. As such, an important consideration in the RAID system is the provision of error detection and correction information, check disks containing redundant information, and crash recovery techniques.
In the Patterson et al. publication five different levels of RAID are described. Level one employs mirrored disks (full redundancy of all disks, both data and check disks), level 2 employs a hamming code for the error correction information to reduce the number of check disks, level 3 employs a single check disk per group of data disks, level 4 employs independent read/write operations wherein the individual transfer information is contained within a single disk unit and is not spread across several disks, and level 5 (RAID5) spreads both the data and the data integrity (parity) information across all disks, including the check disk.